Skip to content

Logging container with syslog-ng

Manage log files centrally

In a context with several different container, it is very hard manage a lot of logs file. Define a single component to collect all the log messages allow to:

  1. Manage logs in a easy way because there are only one source for all the logs produced by containers
  2. if containers crash the logs are still available and no stat data are affected from the disaster ops

Main components of the syslog-ng solution

  • collector:
  • from different sources (files, sockets, pipe)
  • processor:
  • allow to parse amount of data with parser of different types (db parser, python, etc..)
  • apply anonymization technique to protect sensitive information defined in the logs
  • transform the output in a specific format to give at the other infrastructures
  • enrich data with specific additional information
  • filtering
  • avoid not necessary logs using comparisions, wildcard, regular expr and function
  • concatenate all the possibilities with boolean operators
  • storage
  • flat files
  • sent through HTTP(s)
  • big data management (ES, MongoDB, Handhoop and so on)

## Configuration -- attach example of code and trivial user case

## Syslog-ng + Docker - Syslog-ng project started in 1998 - different implementation - with docker is possible define the container that will be used as central server -- approfondire docker infrastructure & container logs journal logs??